<scp>Semi?automatic</scp> coding of <scp>open?ended</scp> text responses in <scp>large?scale</scp> assessments

Background In the context of large-scale educational assessments, effort required to code open-ended text responses is considerably more expensive and time-consuming than evaluation multiple-choice because it requires trained personnel long manual coding sessions. Aim Our semi-supervised method eco (exploring assistant) dynamically supports human raters by automatically a subset responses. Method We map normalized response texts into semantic space cluster vectors based on their similarity. Assuming that similar codes represent semantically responses, we propagate in optimally homogeneous clusters. Cluster homogeneity assessed strategically querying informative presenting them rater. Following each coding, estimates distribution respecting certainty interval assumes if exceeds predefined threshold. If determined certainly comprise all remaining are coded accordingly automatically. evaluated simulation using different data sets. Results With an average miscoding about 3%, reduced 52%. Conclusion Combining advantages automatic produces considerable accuracy reduces effort.